Correlations between hidden units in multilayer neural networks and replica symmetry breaking

We consider feed-forward neural networks with one hidden layer, tree architecture and a fixed hidden-to-output Boolean function. Focusing on the saturation limit of the storage problem the influence of replica symmetry breaking on the distribution of local fields at the hidden units is investigated. These field distributions determine the probability for finding a specific activation pattern of the hidden units as well as the corresponding correlation coefficients and therefore quantify the division of labor among the hidden units. We find that although modifying the storage capacity and the distribution of local fields markedly replica symmetry breaking has only a minor effect on the correlation coefficients. Detailed numerical results are provided for the PARITY, COMMITTEE and AND machines with K=3 hidden units and nonoverlapping receptive fields.Comment: 9 pages, 3 figures, RevTex, accepted for publication in Phys. Rev.

Correlation of internal representations in feed-forward neural networks

Feed-forward multilayer neural networks implementing random input-output mappings develop characteristic correlations between the activity of their hidden nodes which are important for the understanding of the storage and generalization performance of the network. It is shown how these correlations can be calculated from the joint probability distribution of the aligning fields at the hidden units for arbitrary decoder function between hidden layer and output. Explicit results are given for the parity-, and-, and committee-machines with arbitrary number of hidden nodes near saturation.Comment: 6 pages, latex, 1 figur

Odor naming and interpretation performance in 881 schizophrenia subjects: association with clinical parameters

BACKGROUND: Olfactory function tests are sensitive tools for assessing sensory-cognitive processing in schizophrenia. However, associations of central olfactory measures with clinical outcome parameters have not been simultaneously studied in large samples of schizophrenia patients. METHODS: In the framework of the comprehensive phenotyping of the GRAS (Göttingen Research Association for Schizophrenia) cohort, we modified and extended existing odor naming (active memory retrieval) and interpretation (attribute assignment) tasks to evaluate them in 881 schizophrenia patients and 102 healthy controls matched for age, gender and smoking behavior. Associations with emotional processing, neuropsychological test performance and disease outcome were studied. RESULTS: Schizophrenia patients underperformed controls in both olfactory tasks. Odor naming deficits were primarily associated with compromised cognition, interpretation deficits with positive symptom severity and general alertness. Contrasting schizophrenia extreme performers of odor interpretation (best versus worst percentile; N=88 each) and healthy individuals (N=102) underscores the obvious relationship between impaired odor interpretation and psychopathology, cognitive dysfunctioning, and emotional processing (all p<0.004). CONCLUSIONS: The strong association of performance in higher olfactory measures, odor naming and interpretation, with lead symptoms of schizophrenia and determinants of disease severity highlights their clinical and scientific significance. Based on the results obtained here in an exploratory fashion in a large patient sample, the development of an easy-to-use clinical test with improved psychometric properties may be encouraged

Statistical Mechanical Development of a Sparse Bayesian Classifier

The demand for extracting rules from high dimensional real world data is increasing in various fields. However, the possible redundancy of such data sometimes makes it difficult to obtain a good generalization ability for novel samples. To resolve this problem, we provide a scheme that reduces the effective dimensions of data by pruning redundant components for bicategorical classification based on the Bayesian framework. First, the potential of the proposed method is confirmed in ideal situations using the replica method. Unfortunately, performing the scheme exactly is computationally difficult. So, we next develop a tractable approximation algorithm, which turns out to offer nearly optimal performance in ideal cases when the system size is large. Finally, the efficacy of the developed classifier is experimentally examined for a real world problem of colon cancer classification, which shows that the developed method can be practically useful.Comment: 13 pages, 6 figure

Storage capacity of correlated perceptrons

We consider an ensemble of $K$ single-layer perceptrons exposed to random inputs and investigate the conditions under which the couplings of these perceptrons can be chosen such that prescribed correlations between the outputs occur. A general formalism is introduced using a multi-perceptron costfunction that allows to determine the maximal number of random inputs as a function of the desired values of the correlations. Replica-symmetric results for $K=2$ and $K=3$ are compared with properties of two-layer networks of tree-structure and fixed Boolean function between hidden units and output. The results show which correlations in the hidden layer of multi-layer neural networks are crucial for the value of the storage capacity.Comment: 16 pages, Latex2

Landscape-Level Long-Term Biological Research and Monitoring Plan for the Crane Trust

Our obligation is to make sure we are effectively utilizing science to meet the objectives of the Platte River Whooping Crane Maintenance Trust (1981) laid out in its charter “to rehabilitate and preserve a portion of the habitat for Whooping Cranes and other migratory birds in the Big Bend reach of the Platte River between Overton and Chapman (i.e., Central Platte River Valley), Nebraska”. The original declaration is aimed at maintaining “the physical, hydrological, and biological integrity of the Big Bend area as a life-support system for the Whooping Crane and other migratory species that utilize it.” It was clear from the institution’s founding that to accomplish this goal it was necessary to study the effectiveness of land conservation and management actions in providing habitat for Whooping Cranes and other migratory bird species. Quality habitat necessarily comprises all the components that Whooping Cranes and other migratory bird life require to complete their migrations –food and shelter– including nutrient rich diet items such as invertebrates, vascular plants, herpetofauna, fish, and small mammals as well as suitable roosting and foraging locations including wide braided rivers and undisturbed wet meadows (Allen 1952; Steenhof et al. 1988; Geluso 2013; Caven et al. 2019, 2021). Article “A” of the Crane Trust’s (1981) declaration is “to establish a written habitat monitoring plan which can be used to describe change in…[habitat] within the Big Bend of the Platte River…utilized by Sandhill Cranes and Whooping Cranes….” Following initial inventories including avian (Hay and Lingle 1982), vegetation (Kolstad 1981; Nagel 1981), small mammals (Springer 1981), herpetofauna (Jones et al. 1981), insects (Ratcliffe 1981), and fish (Cochar and Jenson 1981), a variety of excellent research has continued at the Crane Trust (https://cranetrust.org/conservation-research/publications/). However, despite the clarity of the Trust’s original declaration, long-term habitat monitoring has not progressed unabated throughout the history of the Crane Trust.https://digitalcommons.unl.edu/zeabook/1130/thumbnail.jp

Whooping Crane (Grus americana) family consumes a diversity of aquatic vertebrates during fall migration stopover at the Platte River, Nebraska

The Aransas-Wood Buffalo population of Whooping Cranes (Grus americana) migrates approximately 4000 km through the central Great Plains biannually, between their breeding and wintering grounds. Whooping Cranes depend on stopover sites to provide secure resting locations and the caloric resources necessary to complete their migration, such as the USFWS-designated critical habitat area in the Central Platte River Valley (CPRV) of Nebraska. This area includes braided river habitat characterized by low-elevation and submerged sandbars, which provide important roosting and foraging opportunities for migrating Whooping Cranes. We used long-range photography, videography, and behavioral scan sampling to document forage items consumed by Whooping Cranes during an 11-day stopover in this area during the fall of 2019. We identified 3 adult-plumage Whooping Cranes and 1 colt consuming 16 individual vertebrates of at least 6 different species during the stopover. In total, we documented Whooping Cranes consuming 7 Channel Catfish (Ictalurus punctatus), 5 ray-finned fish (Actinopterygii), 1 sunfish (Centrarchidae), 1 carp/minnow relative (Cypriniformes), 1 perch relative (Percidae), and 1 Leopard Frog relative (Lithobates sp.). We estimated prey item lengths using the average exposed culmen measurements for adult Whooping Cranes and approximated their nutritional value using log-transformed length–weight regression equations with taxon-specific intercepts and slopes from secondary data sources. We estimated that aquatic vertebrate forage made up a significant portion of Whooping Crane daily energy requirements and provided substantial amounts of calcium, phosphorus, and protein not present at high levels in waste grains also consumed during migration. Additionally, we documented territorial behavior by adult Whooping Cranes during migration and evidence of adults teaching their colt to forage. Our study demonstrates the utility of photography and videography to natural history research and indicates that aquatic vertebrates may be a relatively regular part of Whooping Crane diet in the CPRV. RESUMEN.—La población de grullas trompeteras (Grus americana) de Aransas-Wood Buffalo migra aproximadamente 4000 km dos veces al año a través de las Grandes Llanuras (Great Plains) centrales, entre sus sitios de reproducción y sus sitios de invernada. Las grullas trompeteras dependen de los sitios donde hacen escalas para obtener lugares de descanso y adquirir los recursos calóricos necesarios para completar su migración, tales como, el área de hábitat crítico designada por USFWS en Central Platte River Valley (CPRV) de Nebraska. Esta área cuenta con un hábitat fluvial trenzado caracterizado por bancos de arena sumergidos de baja elevación que constituyen importantes zonas de descanso y alimentación para las grullas trompeteras migratorias. En el otoño de 2019, durante una escala de 11 días, registramos los alimentos consumidos en el área por las grullas trompeteras, utilizando fotografías y videos de largo alcance y análisis conductuales. Identificamos tres grullas trompeteras adultas y una joven consumiendo 16 vertebrados de al menos seis especies diferentes. En total, registramos grullas trompeteras consumiendo siete peces gato americanos (Ictalurus punctatus), cinco peces con aletas radiadas (Actinopterygii), un pez luna (Centrarchidae), un pariente del pez carpa/piscardo (Cypriniformes), un pariente de la perca (Percidae) y un pariente de la rana leopardo (Lithobates sp.). Calculamos la longitud de las presas usando el promedio del tamaño de culmen de las grullas trompeteras adultas, y estimamos el valor nutricional con ecuaciones de regresión de talla-peso transformadas logarítmicamente con intersecciones y pendientes de taxones específicos provenientes de una fuente de datos secundarios. Estimamos que el consumo de vertebrados acuáticos proporciona una parte significativa de las necesidades energéticas diarias de la grulla trompetera, y grandes cantidades de calcio, fósforo y proteínas que no están presentes en altos niveles, en los granos de desecho, que también se consumen durante la migración. Además, documentamos el comportamiento territorial de las grullas trompeteras adultas durante la migración, así como la evidencia de adultos enseñando a sus crías a alimentarse. Nuestro estudio demuestra la utilidad de la fotografía y videografía en la investigación de la historia natural e indica que los vertebrados acuáticos pueden ser parte de la dieta regular de la grulla trompetera en el CPRV

Training a perceptron in a discrete weight space

On-line and batch learning of a perceptron in a discrete weight space, where each weight can take $2 L+1$ different values, are examined analytically and numerically. The learning algorithm is based on the training of the continuous perceptron and prediction following the clipped weights. The learning is described by a new set of order parameters, composed of the overlaps between the teacher and the continuous/clipped students. Different scenarios are examined among them on-line learning with discrete/continuous transfer functions and off-line Hebb learning. The generalization error of the clipped weights decays asymptotically as $exp(-K \alpha^2)$/$exp(-e^{|\lambda| \alpha})$ in the case of on-line learning with binary/continuous activation functions, respectively, where $\alpha$ is the number of examples divided by N, the size of the input vector and $K$ is a positive constant that decays linearly with 1/L. For finite $N$ and $L$, a perfect agreement between the discrete student and the teacher is obtained for $\alpha \propto \sqrt{L \ln(NL)}$. A crossover to the generalization error $\propto 1/\alpha$, characterized continuous weights with binary output, is obtained for synaptic depth $L > O(\sqrt{N})$.Comment: 10 pages, 5 figs., submitted to PR

Replica theory for learning curves for Gaussian processes on random graphs

Statistical physics approaches can be used to derive accurate predictions for the performance of inference methods learning from potentially noisy data, as quantified by the learning curve defined as the average error versus number of training examples. We analyse a challenging problem in the area of non-parametric inference where an effectively infinite number of parameters has to be learned, specifically Gaussian process regression. When the inputs are vertices on a random graph and the outputs noisy function values, we show that replica techniques can be used to obtain exact performance predictions in the limit of large graphs. The covariance of the Gaussian process prior is defined by a random walk kernel, the discrete analogue of squared exponential kernels on continuous spaces. Conventionally this kernel is normalised only globally, so that the prior variance can differ between vertices; as a more principled alternative we consider local normalisation, where the prior variance is uniform

Ocean current connectivity propelling the secondary spread of a marine invasive comb jelly across western Eurasia

Aim: Invasive species are of increasing global concern. Nevertheless, the mechanisms driving furtherdistribution after the initial establishment of non-native species remain largely unresolved, especiallyin marine systems. Ocean currents can be a major driver governing range occupancy, but this hasnot been accounted for in most invasion ecology studies so far. We investigate how well initialestablishment areas are interconnected to later occupancy regions to test for the potential role ofocean currents driving secondary spread dynamics in order to infer invasion corridors and thesource–sink dynamics of a non-native holoplanktonic biological probe species on a continental scale.Location: Western Eurasia.Time period: 1980s–2016.Major taxa studied: ‘Comb jelly’ Mnemiopsis leidyi.Methods: Based on 12,400 geo-referenced occurrence data, we reconstruct the invasion historyof M. leidyi in western Eurasia. We model ocean currents and calculate their stability to match thetemporal and spatial spread dynamics with large-scale connectivity patterns via ocean currents.Additionally, genetic markers are used to test the predicted connectivity between subpopulations.Results: Ocean currents can explain secondary spread dynamics, matching observed range expansionsand the timing of first occurrence of our holoplanktonic non-native biological probe species,leading to invasion corridors in western Eurasia. In northern Europe, regional extinctions after coldwinters were followed by rapid recolonizations at a speed of up to 2,000 km per season. SourceJASPERS ET AL. | 815areas hosting year-round populations in highly interconnected regions can re-seed genotypes overlarge distances after local extinctions.Main conclusions: Although the release of ballast water from container ships may contribute tothe dispersal of non-native species, our results highlight the importance of ocean currents drivingsecondary spread dynamics. Highly interconnected areas hosting invasive species are crucial forsecondary spread dynamics on a continental scale. Invasion risk assessments should considerlarge-scale connectivity patterns and the potential source regions of non-native marine species